Exploring and Optimizing Reinforcement Learning Algorithms in the Frozen Lake Environment

Analysis of RL algorithms in deterministic and stochastic Frozen Lake environments, with hyperparameter optimization via Optuna.

This project evaluates reinforcement learning algorithms in the Frozen Lake environment, under both deterministic and stochastic settings.
Algorithms studied include Monte Carlo, Sarsa, Expected Sarsa, Q-Learning, and Double Q-Learning.

A key focus is hyperparameter optimization with Optuna, applied to Q-Learning to improve learning efficiency.

Key results:
- Q-Learning and Double Q-Learning performed robustly in stochastic scenarios
- Discount rate and learning rate were critical for convergence and stability
- Optuna tuning enhanced Q-Learning’s efficiency across environments
- Findings provide guidance for RL applications in grid-world tasks

Read full-text on ResearchGate

Exploring deterministic and stochastic Frozen Lake environments with RL